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ABSTRACT 


We  have  obtained  a  number  of  results  pertaining  to  image  compression,  robust 
estimation,  and  robust  signal  detection.  All  of  this  work  has  admitted  the  presence 
of  data  whose  statistics  are  imperfectly  known.  Our  results  have  featured  adaptivity, 
flexibility,  and  nontraditional  approaches.  In  order  to  employ  more  realistic  statisti¬ 
cal  models,  we  have  directed  our  research  to  admit  nonstationarity  and  dependency.  , 
Much  of  our  work  in  robust  estimation  and  detection  has  employed  a  geometric  ap¬ 
proach  which  we  have  originated  in  past  research.  Our  geometric  techniques  provide 
a  quantitive  way  to  measure  the  degree  of  robustness,  thus  offering  the  designer  more 
flexibility  in  meeting  the  performance/robustness  needs  of  the  user.  Our  results  in¬ 
clude  generalized  robustness  criteria  involving  curvature  as  well  as  manifold  slope,  as 
well  as  generalized  nonlocal  robustness  criteria  which  supersede  prior  nonlocal  criteria 
based  on  the  '‘worst  case”  perspective.  In  addition,  we  have  applied  the  geometric 
perspective  to  show  how  linear  estimation  algorithms  can  be  modified  to  optimize  a 
weighted  combination  of  performance  and  robustness,  thus  offering  the  user  the  option 
of  selecting  various  performance/robustness  combinations  as  deemed  appropriate  for  a 
specific  application. 
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SUMMARY  OF  RESEARCH 


The  research  supported  by  Grant  AFOSR-87-0087  has  been  primary  concerned 
with  robust  signal  detection  and  estimation,  although  some  effort  has  also  been  directed 
toward  image  compression.  The  latter  research  area  will  be  addressed  first. 

A  recently  developed  technique  for  image  compression  is  Block  Truncation  Coding 
(BTC),  a  method  which  is  easy  to  implement  and  often  possesses  good  performance 
characteristics,  relative  to  other  approaches,  even  in  the  presence  of  many  channel 
errors.  In  addition,  the  BTC  algorithm  does  not  depend  on  knowledge  of  the  specific 
underlying  statistical  distributions  but  rather  on  sample  moments.  The  basic  BTC 
algorithm  employs  a  two  level  quantizer  whose  output  levels  are  obtained  by  matching 
sample  moments.  The  image  is  divided  into  blocks  of  pixels,  and  the  BTC  algorithm  is 
applied  block  by  block.  In  some  past  work  we  have  extended  the  basic  BTC  technique 
to  a  much  more  general  setting  which  admits  various  options  of  implementation  with 
the  potential  for  improved  performance.  In  our  current  work,  we  note  that  since  various 
particular  options  seem  most  appropriate  to  certain  image  characteristics,  the  uniform 
application  of  one  specific  option  across  the  entire  image  (as  in  earlier  work)  may  not 
lead  to  best  performance.  Our  current  work  illustrates  an  adaptive  modification  of  the 
earlier  BTC  approach,  which  admits  adjusting  the  algorithm  on  a  block  by  block  basis. 
While  doing  so,  the  algorithm  chooses  among  various  optional  sample  moments,  as  well 
as  the  options  of  “black  on  white”  (higher  numbers  are  assigned  to  lighter  gray  levels) 
or  "white  on  black”  (higher  numbers  are  assigned  to  darker  gray  levels).  We  then  show 
by  example  that  such  block  by  block  adaptivity  can  lead  to  improved  performance.  Our 
work  also  shows  that  for  the  example  images  considered,  the  option  employed  across 
the  entire  image  by  the  basic  BTC  algorithm  is  chosen  by  the  adaptive  algorithm  for 
less  than  ten  percent  of  the  pixel  blocks,  thus  illustrating  the  appeal  of  our  adaptive 
approach.  These  results  are  delineated  in  #3  of  the  publication  list. 

A  major  thrust  of  our  effort  has  been  in  the  direction  of  robust  signal  detection. 
Some  of  our  work  has  employed  classical  saddlepoint  methodology  to  impart  robust  ness. 
In  some  recent  work  we  investigate  a  situation  where  inexact  knowledge  of  the  statistics 
of  the  noise  is  present  for  the  discrete  time  detection  of  signals  in  i.i.d.  non-Gaussian 
noise.  In  earlier  work  employing  the  canonical  form  of  the  locally  optimal  detector, 
some  othei  authors  showed  how  the  design  of  the  asymptotically  robust  detector  for  the 
Huber-Tukey  mixture  family  of  noises  could  be  obtained.  As  might  have  been  expected. 
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the  results  led  to  censoring  the  factors  of  the  test  statistic  to  impart  robustness,  which 
led  to  the  employment  of  a  detector  nonlinearity  which  limited  observations  of  large 
magnitude.  However,  this  work  did  not  take  into  account  the  common  situation  where 
more  is  known  about  a  noise  density  near  the  origin  than  on  the  tails.  In  fact,  the  noise 
often  arises  in  practice  from  the  sum  of  a  number  of  “nearly”  independent  sources,  and 
thus  frequently  the  noise  density  resembles  the  Gaussian  near  the  orgin  but  may  differ 
markedly  on  the  tails.  Such  knowledge  had  the  potential  to  be  exploited  for  improved 
performance,  where  the  robustness  is  imparted  to  account  for  our  lack  of  knowledge 
of  the  tails  of  the  noise  density.  We  have  investigated  this  opportunity  and  obtained 
results  which  specify  an  asymptotically  robust  detector  for  the  situation  where  the  noise 
density  is  known  on  an  interval  about  the  origin  for  the  case  where  the  signal  is  modeled 
as  a  random  process.  We  then  show  by  way  of  example  that  our  robust  detectors  offer 
improved  performance  over  the  previous  detectors;  in  fact,  the  improvement  is  quite 
dramatic  in  certain  cases.  These  results  are  delineated  in  #2  of  the  publication  list. 

While  classical  saddlepoint  techniques  can  be  useful  in  investigating  certain  ques¬ 
tions  pertaining  to  robustness,  we  believe  that  the  exceedingly  heavy  reliance  placed 
by  researchers  on  these  techniques  stems  more  from  inertia  and  the  apparent  lack  of 
suitable  alternatives,  rather  than  a  universal  recognition  that  saddlepoint  techniques 
are  ultimately  suitable  in  all  important  respects.  In  our  recent  work,  a  desirable  al¬ 
ternative  has  been  developed  which  approaches  the  concept  of  robustness  through  a 
perspective  based  on  differential  geometry.  While  this  approach  has  found  application 
quite  naturally  to  robust  hypothesis  testing,  the  core  of  signal  detection,  the  approach 
also  has  the  advantage  of  possessing  the  potential  for  wider  domains  of  application.  The 
geometric  techniques  employed  allow  the  computation  of  a  quantitative  measure  of  the 
degree  of  robustness  of  a  given  detector  subject  to  a  wide  variation  of  the  underlying 
statistical  distribution  function  about  a  nominal  distribution.  This  new  approach  pos¬ 
sesses  distinct  advantages  when  compared  to  classical  saddlepoint  criteria,  which  are 
inherently  nonquantitative  and  are  relevant  to  only  a  few  restricted  canonical  regions 
of  admissible  variation  about  a  nominal  distribution.  Moreover,  the  potential  sacrifices 
in  performance  implied  by  the  choice  of  a  classical  saddlepoint  robust  detector  may  be 
serious;  some  of  our  recent  work,  which  completely  specifies  the  classical  saddlepoint 
robust  detector  for  a  signal  in  nominally  Laplace  noise,  also  shows  that  substantial 
sacrifices  in  performance  are  experienced.  Our  approach  admits  an  analysis  based  on 
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the  quantitative  tradeoff  of  performance  and  robustness,  thus  providing  the  user  with 
a  much  more  flexible  design  procedure  than  that  based  on  the  classical  techniques. 

The  aforementioned  geometric  approach  introduced  by  us  has  provided  quantitative 
measures  of  the  degree  of  robustness  for  both  the  local  case,  where  the  results  have  ap¬ 
plication  only  for  small  neighborhoods  of  distributions  near  the  nominal,  and  the  more 
general  nonlocal  case.  These  results  have  been  applied  to  compare  the  robustness  of 
a  Neyman-Pearson  optimal  detector  with  a  detector  which  employs  a  censored  version 
of  the  Neyman-Pearson  optimal  nonlinearity,  the  form  of  the  latter  (hereafter  called 
Huber-type)  arising  when  classical  saddlepoint  methodology  is  utilized.  In  addition, 
the  optimization  of  a  weighted  linear  cost  criterion  of  performance  and  robustness  has 
been  undertaken,  with  the  results  specifying  the  optimal  censoring  height  of  the  de¬ 
tector  nonlinearity.  The  accomplishment  of  such  an  undertaking  would  not  have  been 
an  appropriate  goal  under  the  nonquantitative  classical  saddlepoint  design  methods, 
and  the  success  of  our  geometric  approach  in  addressing  such  goals  has  encouraged  fur¬ 
ther  investigations.  The  development  of  the  geometric  approach,  together  with  several 
applications,  has  been  summarized  in  #13  of  the  publication  list. 

Because  our  results  have  disclosed  specific  disadvantages  as  well  as  advantages  of  the 
Huber-type  detector,  we  have  recently  investigated  a  more  general  nonstandard  family 
of  robust  detectors;  this  family  offers  a  more  varied  range  of  performance  and  robustness 
options.  Our  work  considers  both  the  local  nonlocal  situation,  and  we  show  that,  by 
proper  choice  of  the  parameters  associated  with  the  nonstandard  family,  a  much  richer 
mix  of  performance  and  robustness  is  possible.  These  results  are  delineated  in  #1  of 
the  publication  list. 

The  aforementioned  results  using  our  geometric  techniques  have  employed  an  as¬ 
sumption  of  i.i.d.  data.  While  in  some  cases  it  may  be  appropriate  to  employ  a  suffi¬ 
ciently  slow  sampling  rate  to  justify  an  assumption  of  independence,  it  is  precisely  in 
this  kind  of  circumstance  that  stationarity  assumptions  become  less  credible.  We  there¬ 
fore  have  recently  considered  removing  the  stationarity  assumption  in  our  geometric 
approach,  and  we  have  found  that  results  analogous  to  the  i.i.d.  case  are  obtainable  for 
both  the  local  and  nonlocal  situation  if  one  assumes  independence  without  stationarity. 
One  particularly  interesting  consequence  of  these  results  is  that  for  many  of  the  exam¬ 
ples  considered,  the  Huber-type  robust  detector  becomes,  like  the  Neyman-Pearson 
optimal  detector,  completely  unrobust  as  the  number  of  samples  approaches  infinity. 
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The  robustness  advantages  of  the  Huber-type  detector  thus  atrophy  and  approach  zero 
for  large  numbers  of  samples,  increasing  the  appeal  of  alternative  nonstandard  families 
of  robust  detectors.  We  have  also  undertaken  an  accompanying  preliminary  study  of 
the  effects  of  dependency,  and  our  work  shows  how  a  measure  of  robustness  can  be  de¬ 
veloped  for  the  case  when  the  joint  distribution  function  is  parameterized  by  unknown 
parameters.  One  surprising  consequence  of  these  results  is  that  we  provide  an  example 
of  a  parameterized  Gaussian  family  of  joint  distributions  for  which  the  linear  detector 
is  more  robust  than  a  corresponding  censored  linear  (Huber-type)  detector.  Employ¬ 
ment  of  a  Huber- type  detector  may  therefore  do  more  harm  than  good  if  dependency 
is  expected,  and  thus  the  classical  robust  detector  is  not  at  all  “robust”  to  possible  per¬ 
turbations  in  the  independence  assumption.  We  also  have  investigated  more  general 
types  of  dependency,  and  our  work  shows  that  the  definition  for  the  robustness  measure 
which  is  analogous  to  the  corresponding  definition  in  the  independent  case  does  not 
lead  to  a  meaningful  result.  This  difficulty  arises  because  the  required  integrals  fail  to 
converge,  presumably  because  there  are  too  many  allowable  variations  in  the  multidi¬ 
mensional  distribution  functions  under  consideration.  A  modification  of  our  techniques 
for  the  general  dependent  case  would  thus  be  desirable.  The  aforementioned  results  are 
delineated  in  #4  of  the  publication  list. 

In  some  recent  work,  we  have  achieved  a  major  breakthrough  be  accomplishing 
the  aforementioned  modification  of  our  techniques,  thus  admitting  application  to  the 
general  dependent  case.  Our  results  provide  a  natural  quantitative  measure  of  the 
robustness  of  a  signal  detector  with  dependent  data;  this  measure  is  sensitive  to  essen¬ 
tially  arbitrary  perturbations  in  an  underlying  joint  distribution  away  from  the  nominal. 
Our  results  show,  for  example,  that  the  presence  of  residual  dependency  can  result  in 
a  reduction  of  robustness;  in  particular,  this  reduction  is  approximately  50%  for  the 
linear  detector.  Our  work  therefore  shows,  for  the  first  time,  precisely  how  serious  the 
consequences  of  residual  dependency  can  be.  These  results  are  delineated  in  #9  of  the 
publication  list. 

In  some  other  recent  work,  we  have  shown  how  our  geometric  techniques  apply  to  a 
variety  of  signal  processors.  This  work  also  includes  results  for  measuring  the  robustness 
of  a  nonstandard  family  of  signal  detectors  featuring  nonempty  “negative  boundaries". 
This  family  is  seen  to  offer  the  potential  for  enhanced  robustness  in  certain  situations 
over  standard  classical  saddlepoint  robust  detectors  and  other  detectors  possessing 


empty  “negative  boundaries”,  thus  opening  the  door  for  a  more  detailed  analysis.  These 
results  are  delineated  in  #6  and  #7  of  the  publication  list. 

The  aforementional  work  involving  “negative  boundaries”  has  recently  been  ex¬ 
tended  by  us  to  include  a  comparison  of  the  standard  family  of  classical  robust  de¬ 
tectors  to  an  important  subfamily  of  the  aforementioned  nonstandard  family.  Using 
an  optimized  cost  criterion  that  reflects  the  contribution  of  both  performance  and  ro¬ 
bustness,  we  show  that  when  performance  is  emphasized  then  the  standard  family  is 
most  suitable;  however,  when  robustness  is  emphasized  then  the  ncnstandard  family 
is  preferable.  Our  work  thus  allows  the  user  to  select  a  detector  which  most  closely 
matches  the  desired  performance/robustness  mix.  These  results  are  delineated  in  #8 
of  the  publication  list. 

Our  current  work  also  is  directed  toward  the  area  of  robust  estimation.  We  have 
sought  to  apply  the  aforementioned  geometric  techniques  to  estimation  as  well  as  detec¬ 
tion,  and  we  have  shown  that  this  route  can  lead  to  success.  As  with  signal  detection, 
we  desire  a  quantitative  way  to  measure  the  degree  of  robustness  of  an  estimator  sub¬ 
ject  to  lack  of  knowledge  of  the  underlying  statistical  distributions.  There  appears  to 
be  more  than  a  passing  kinship  between  the  areas  of  signal  detection  and  estimation; 
both  have  optimal  forms  which  arise  under  Radon-Nikodym  derivatives,  and  both  have 
robust  forms  which  can  be  seen  to  arise  under  our  geometric  methodology.  We  have 
recently  undertaken  a  quantitative  analysis  of  the  application  of  this  methodology  to 
robust  parameter  estimation,  where  general  performance  measures  (including,  but  not 
limited  to.  mean  square  error)  as  well  as  nonstationary  data  are  admitted.  Our  results 
show  not  only  why  censoring  again  can  be  helpful  in  imparting  robustness,  but  also 
how  limitations  involving  censoring  can  arise,  including  the  atrophy  of  robustness  for 
large  numbers  of  samples.  As  with  robust  signal  detection,  our  results  facilitate  the 
design  of  an  estimator  which  trades  off  performance  versus  robustness  subject  to  a  cost 
criterion  of  interest  of  the  user.  These  results  are  delineated  in  #5  of  the  publication 
list. 

In  conjunction  with  the  aforementioned  breakthrough  admitting  general  dependent 
data  in  signal  detection,  we  have  also  employed  analogous  modifications  to  our  work  in 
parameter  estimation  to  allow  measuring  the  robustness  of  an  estimator  of  a  random 
variable.  Our  work  shows  that  the  choice  of  performance  fidelity  criterion  is  crucial. 
For  example,  we  find  that  for  many  popular  error  criteria  (e.g.  mean  square  error)  any 
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admissable  estimator  is  completely  unrobust.  On  the  other  hand,  we  also  show  that 
there  exists  a  wide  family  of  performance  criteria  for  which  some  degree  of  robustness 
can  be  achieved.  Somewhat  surprisingly,  in  this  case  the  best  estimator  has  been  shown 
by  us  to  also  often  be  the  most  robust.  The  conditional  expectation  estimator,  known  to 
be  an  optimal  estimator  under  a  variety  of  error  criteria,  thus  can  be  either  completely 
unrobust  or  optimally  robust,  depending  on  the  choice  of  error  criterion.  The  consider¬ 
ation  of  alternative  error  criteria  to  the  popular  mean  square  error  crit"-:,on  is  therefore 
highly  desirable.  We  have  also  exhibited  success  in  generalizing  our  oU  *  -  geometric 
approach,  which  is  local  in  nature,  to  the  nonlocal  arena  by  employing  a  “worst  case" 
perspective  analogous  to  that  used  with  some  success  by  the  c'assical  methodology. 
Our  new  nonlocal  techniques,  however,  admit  the  consideration  of  essentially  arbitrary 
regions  about  the  nominal  distribution,  and  can  be  used  to  draw  nonlocal  conclusions 
for  a  variety  of  important  robustness  questions,  including  how  to  control  detector  false 
alarm  rate  as  the  underlying  distribution  varies  over  a  region  containing  the  nominal. 
These  results  are  delineated  in  #9  of  the  publication  list. 

An  important  family  of  estimators  of  a  random  variable  is  the  family  of  linear  es¬ 
timators,  which  includes  such  popular  algorithms  ao  the  Kalman-Bucy  filter.  Using 
the  geometric  approach,  we  have  recently  shown  that,  under  fidelity  criteria  yielding 
nonzero  robustness,  the  linear  estimator  which  yields  the  best  performance  does  not 
necessarily  offer  maximal  robustness.  This  situation  is  therefore  analogous  to  that  for 
signal  detection,  where  optimal  performance  (via  a  Neyman-Pearson  test)  and  optimal 
robustness  do  not  necessarily  coincide.  The  user  must,  in  such  situations,  judiciously 
trade  off  the  two  desirable  attributes  of  performance  of  robustness,  and  we  have  accord¬ 
ingly  shown  how  to  accomplish  this  by  designing  an  appropriate  linear  estimator  which 
optimizes  a  corresponding  cost  criterion  weighting  the  two  attributes.  Our  results  allow 
the  user  to  specify  robustness/performance  mixes  of  100%/0%  all  the  way  to  0%/100%. 
These  results  are  delineated  in  #12  of  the  publication  list. 

We  have  also  developed  a  new  approach  toward  nonlocal  robustness,  which  includes, 
but  is  not  limited  to,  the  “worst  case"  methods  used  by  the  classical  approach  and  our 
geometric  approach.  This  generalized  nonlocal  robustness  criterion  recognizes  that  the 
actual  underlying  distribution  will  very  likely  be  neither  nominal  (as  employed  under 
optimal  methods)  nor  least  favorable  (as  employed  under  prior  nonlocal  robustness 
methods).  Our  generalized  approach  admits  the  employment  of  arbitrary  weighted 
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contributions  of  all  distributions  in  the  admissable  family,  and  not  just  the  least  favor¬ 
able.  For  example,  a  user  might  expect  a  greater  likelihood  that,  the  actual  distribution 
would  be  closer  to  the  nominal  than  the  least  favorable,  and  might  wish  to  incorporate 
such  an  assumption  into  the  design  of  a  robust  scheme.  Our  work  develops  closed  form 
expressions  for  generalized  nr  local  robustness  measures  with  application  within  both 
signal  detection  and  estimation,  using  both  linear  and  nonlinear  weighting  of  the  con¬ 
tributions  of  all  distributions  within  the  admissable  family.  These  results  are  delineated 
in  #11  of  the  publication  list. 

All  of  the  aforemen noned  work  which  employs  our  recently  originated  geometric  ap¬ 
proach  has  been  based  on  “first  order"  measures  of  robustness,  wherein  manifold  slope 
is  the  key  quantity.  While  such  measures  are  suitable  in  many  ways,  we  have  found 
that  on  occasion  two  different  detectors  or  estimators  can  have  the  same  robustness 
under  the  operative  “first  order"  measure.  In  such  cases,  more  delicate  measures  are 
required,  and  we  have  succeeded  in  developing  such  measures  through  the  introduc¬ 
tion  of  a  suitable  choice  of  manifold  curvature.  Our  work  shows  that  these  resultant 
measures  reduce  to  two  dominating  terms,  with  the  first  yielding  our  existing  operative 
"first  order"  measure,  together  with  an  additional  “second  order”  term  which  allows 
differentiating  between  two  detectors  or  estimators  which  have  the  same  “first  or  ler" 
robustness.  We  have  also  shown  by  way  of  example  that  nonlinear  processing  of  the 
data  together  with  censoring  can  offer  a  23%  improvement  in  “second  order”  robustness 
when  compared  to  linear  processing  with  censoring,  even  though  both  the  linear  and 
nonlinear  processing  algorithms  possess  the  same  "first  order”  robustness.  The  em¬ 
ployment  of  "second  order"  robustness  is  thus  a  useful  auxiliary  tool  in  the  assessment 
of  the  sensitivity  of  an  algorithm  to  inexact  statistical  knowledge.  These  results  are 
delineated  in  #10  of  the  publication  list. 
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